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ABSTRACT 



Guilford County School District is the third largest school 
district in North Carolina. It includes urban, suburban, and rural 
neighborhoods and a full range of income levels with a population that is 
approximately 60% white and 40% minority, predominantly African American. 

When three previous school districts merged to form Guilford County Schools, 
the Middle School Program was initiated. This program called for evaluation 
annually. In 1995-96, evaluation efforts consisted of the administration of a 
survey to all students, teachers, and parents. The survey was repeated in the 
second year to establish a firm baseline, and the design of a new evaluation 
plan began in the second year. In 1997-98 schools were to provide input to 
the formative evaluation program, and in 1998-99 the formative evaluation 
plans were to be carried out. Then in the 1999-2000 school year, the 
administration of the survey to parents, students, and teachers is to be 
repeated. In 1995-96, 857 middle school teachers, 6,938 parents, and 11,281 
students from the district's 17 middle schools responded to the survey, which 
focused on strengths and weaknesses of the school and suggestions for 
improvement. In 1997 responses were received from 878 teachers, 6,429 
parents, and 11,774 students. Overall results were positive, with few changes 
noted in the responses between the 2 years. In each year, schools received 
their survey item data in tabular and graphic form to use in their School 
Improvement Plans. Anecdotal reports by principals suggested that these 
reports were very helpful . Two appendixes contain sample reports from both 
years. (Contains one reference.) (SLD) 
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A Systems Approach to Middle School Evaluation: Guilford County 
Schools’ Formative Approach 

ON 

fo Ward, Martha; Director of Assessment and Evaluation, Guilford County 

^ Schools, Greensboro, NC 

Q Paper presented at the annual conference of the American Educational 

Research Association, San Diego, CA, April 1998 

Background on Guilford County Schools and the Middle School 
Program 

Guilford County Schools covers approximately 600 square miles in the 
Piedmont region of North Carolina. With a current student enrollment of 
approximately 60,000 and 8,000 employees, the system ranks as about the 
60*h largest nationally, and third largest in North Carohna. The current 
county-wide system replaced three former school districts — two city 
(Greensboro & High Point) and one county (Guilford) - which were fully 
merged as of the 1993-94 school year. Guilford County includes urban, 
suburban, and rural neighborhoods and a full range of income levels. The 
student population is approximately 60 percent white and 40 percent 
minority, predominantly African American. There are increasing proportions 
of Asian students and students with hmited Enghsh proficiency. 

At the time of merger, the Guilford County Schools Board of Education 
committed to unified programs in a number of areas, one of which was the 
Middle School Program. The Middle School Concept had been implemented 
in each of the former districts, but with varying degrees of success. A process 
over the course of several years ensued. Fundamental to the process was the 
creation of a Middle School Task Force composed of educators and community 
members, which was extensively involved in the development of the Middle 
School Program. 

The plan for Guilford County Schools was congruent with the central 
components of the middle school concept (George and Alexander, 1993). The 
seven central elements are: 
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• focus on academic achievement in the core curriculum, 

• daily teacher advisory, 

• team organization in every grade, 

• flexible block scheduling, 

• an expanded menu of electives and student activities, and 

• differentiated instruction, and heterogeneous grouping in 
science and social studies in all schools. 
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The Board of Education of the newly merged system formally adopted the 
Middle School Program and called for an annual evaluation. The program 
was put into place in 1995-96, and the Guilford County Schools Office of 
Assessment and Evaluation undertook the evaluation, in cooperation with 
the Office of Secondary Achievement. 

Methods 

Evaluation Process and Timeline 

The mandate by the Board of Education for an annual evaluation required 
fast planning and implementation for the first year. It was determined that a 
survey of students, teachers, and parents would occur as a first step. After 
the first survey results were made available, the need to repeat the survey for 
a second year in order to establish a firm basehne was expressed. At the 
same time, there was a desire to broaden the scope of the evaluation to 
include data collection instruments in addition to the survey, and to involve 
the schools in the formative evaluation of their own efforts. For these 
reasons the following timeline for the evaluation evolved: 

1995- 96 Administration of the Guilford County Schools 

Middle Schools survey to all students, teachers, 
and parents 

1996- 97 Administration of the Guilford County Schools 

Middle Schools survey to all students, teachers, 
and parents 

Begin design of new evaluation plan 

1997- 98 Schools determine needs and provide input to the 

school’s formative evaluation plan for 1997-98 and 

1998-99 

1998- 99 Schools’ formative evaluation plans are carried out 

1999- 00 Administration of the Guilford County Schools 

Middle Schools survey to all students, teachers, 
and parents 



Survey Development 

Increased academic achievement was already a major focus of the 
Superintendent and the Board of Education, so academic achievement 
per se was not included in the survey or later evaluation plan. Survey 
questions focused on: 
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• The Middle School Concept; 

• Instruction and Learning, e.g. expectations, effort; 

• School Climate; 

• Parent Involvement; 

• “Grading” the School. 

A matrix was developed with the key content of the survey comprising one 
axis and the three target audiences on the other. Items were developed for 
each audience, if they were viewed as “competent” to respond. For example, 
all respondents were asked questions relating to parent involvement, while 
parents and students were not asked questions about aspects of the Middle 
School Concept such as Team Planning. 

Drafts of the survey were reviewed by Central Office staff including 
curriculum specialists, the Superintendent’s Cabinet, the Board of Education, 
student teachers, parents, and several students. Appropriate changes were 
made. 

Open-Ended Survey Items 

In addition to the multiple-choice items, in 1996 three open-ended questions 
were asked: 

1. In your opinion, what were the strengths of this school? 

2. In you opinion, what are the biggest problems facing this 
school? 

3. What are your suggestions for solving some of these problems 
or what other improvements would you suggest for this 
school? 

Question 3 was ehminated in 1997, since it produced responses that 
reiterated responses to the first two questions. 

Survey Administration 

During 1996-97 the survey was conducted in the 17 GCS Middle Schools 
during April and May. The survey was repeated in a very similar manner in 
April 1997. The three survey forms (one for each of the three target groups) 
were distributed through the schools. All individuals in these groups had the 
opportunity to respond to the survey. 

School identification codes were pre-coded onto the machine-readable survey 
forms. The three sets of survey forms were shipped to the schools, with 
overall instructions to the principal for the distribution of materials. 
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Principals were instructed to administer the survey to teachers as a group 
(for example during a staff meeting) prior to distributing student and parent 
forms to teachers. Principals were provided with precise instructions about 
informing teachers about the survey, and for conducting the survey. Surveys 
were collected immediately and sealed in an envelope for return to the Office 
of Assessment and Evaluation. 

Teachers were provided with precise instructions about informing students 
about the survey, and for conducting the survey. Surveys were collected 
immediately and sealed in an envelope for return to the Office of Assessment 
and Evaluation. 

The parent survey packets included the survey, instructions, and an envelope 
which was to be sealed prior to returning the survey to the school. Schools 
had the capacity to produce maOing labels electronically for each student in 
the school. School staff affixed the labels onto the packets which had been 
prepared for each group of students. Teachers sent the packets home with the 
students, and kept a roster of those students who returned the surveys. 
Principals were encouraged to find creative ways to encourage students to 
return the surveys. At the end of a three week period, all survey materials 
were returned to Assessment and Evaluation for scanning. 

Results 

Response Rates 

In 1996 Survey responses were received from 857 middle school teachers, 
6,938 parents, and 11,281 students. This translates into approximately the 
following response rates: 

• 100 percent of students 

• 100 percent of teachers 

• 62 percent of parents 

In 1997, survey responses were received from 878 middle school teachers, 
6,429 parents, and 11,774 students. This translates into approximately the 
following response rates: 

• 100 percent of students 

• 100 percent of teachers 

• 55 percent of parents 

In essence, the student and parent groups were “captive audiences”. The 
response rate for parents was calculated using the number of responses 
received divided by the April Average Daily Attendance. Exact number of 
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households is somewhat smaller than the number of students, since some 
households have more than one child in middle school. 

No attempt was made to adjust the parent sample for representativeness. 
While 34.1 percent of students reported their ethnic group as Black/African 
American, 28.0 percent of parents did so. Additionally, 72.8 percent of the 
parent respondents were female. 

Analysis and Reporting to Schools 

Responses to each item were weighted from +2 to -2, with neutral responses 
receiving a score of 0. With this weighting system, averages of 0 represented 
neutrality, positive numbers represented positive responses, and negative 
number represented negative responses. The standard deviations were also 
reported and interpreted as an index of agreement. 

Tabular and graphic results were summarized for the school system as a 
whole, and for each school. (Appendix 1 provides samples.) The second year, 
comparative data from 1996 was included (see Appendix 2.) 

Overall Results 

Overall the results were very positive, although areas needing improvement 
were noted. System-wide, there were few changes in the responses between 
1996 and 1997. A comparison of 1996 and 1997 results revealed improved 
ratings in a number of areas such as teacher advisory, school-within-a-school, 
and school safety. There was a great deal of variation among schools, both in 
terms of the nature of the responses, and in changes from 1996 to 1997. 

The responses can be grouped into five broad categories, shown below. (The 
questions shown are illustrative rather than complete.) The percentages 
shown are from 1997. 

1. The Middle School Concept (includes Advisor/Advisee classes, flexible 
block scheduhng, a thematic interdisciphnary curriculum and 
interdisciplinary teams, electives, clubs, and athletic activities) 

• Overwhelming, teachers agreed that the administrative team is effective 
in defining the middle school concept (90%), and that their school staff is 
effective in its implementation (91%). 

• With respect to interdisciplinary teams, teachers agreed that their team 
worked effectively together (76%), and reported that teams have goals and 
objectives for the year (78%), and plan for interdisciphnary instruction 
(66%). Teachers agreed that teams helped to improve students’ sense of 
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belonging (91%). Seventy five percent of students reported that they were 
on a “good team”. 

• Most teachers (67%) believed that flexible block scheduhng is an 
important component of the Middle School Program, that it improves 
student achievement (71%), student attitude about school (66%), and the 
teacher’s attitude toward teaching (70%). 

• While 92% of the teachers agreed that elective/encore courses are an 
important component of the middle school plan, only 54% reported that 
elective teachers are a part of the team, and 63% reported that core and 
elective teachers work together to address student needs. The majority of 
teachers and parents were satisfied with the number and variety of 
elective classes offered (over 77% in each case), while students were less 
satisfied (between 60% and 70%). 

• Teachers (72%) generally agreed about the importance of the 
Advisor/Advisee program, and 51% followed a curriculum. The majority of 
students reported positive feehngs about the program, but 44% reported 
doing homework during A/A class. 

2. Instruction and Learning (includes expectations, effort, diversity and 

equity, student success, and the use of technology and media) 

• Teachers were confident that they were meeting the academic needs of 
their students (89%), and that they were emplo 5 dng instructional 
strategies that accommodate the diverse learning styles of students (95%). 
Teachers reported that they provided a variety of teaching strategies and 
learning activities in their classes (96% and 94%, respectively). Teachers 
expected aU of their students to be successful (98%). Parents agreed that 
their children were given challenging work in both core and elective 
classes (about 80 in core classes, and 78% in electives). 

• Over 77% of teachers agreed that heterogeneous grouping is an important 
component of middle schools and that this practice improves students’ 
attitude toward school. However, there is a clear need to provide 
continued training in how to differentiate instruction in heterogeneous 
classes. For example, 63% of teachers reported a behef that students of 
all abilities succeed in heterogenously grouped classes. Of parents, 63% 
stated that teachers were able to meet the needs of children in 
heterogeneous classes. 

• Students perceive a high level of “academic press” from their teachers. 
They found their mathematics classes (78%), and other core classes such 
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as science and social studies to be challenging (around 70%). A majority 
agreed that they were learning the things they will need in the future 
(73%). 

• Parents reported that their children were enthusiastic about learning at 
the school (76%), and were getting the skills they need (80%). 

• Teachers reported that the Media Center accommodated the needs of 
teachers (91%) and students (90%). Students enjoyed using the Media 
Center (76%) and felt welcome there (73%). 

3. School Climate (includes behavior, disciphne, school safety and 

cleanhness, respect and caring, and support programs) 

• Teachers (89%) and students (61%) felt safe in their school. The majority 
of parents reported that their children felt safe (80%). Students reported 
that students at the school fight a lot (55%), and do not follow school rules 
(54%). Many teachers (61%) and fewer parents (30%) reported that 
student misbehavior was a problem at the school. Only about 50% of 
teachers reported that in-school-suspension and school-within-a-school 
improved student disciphne. With respect to the school being clean and 
comfortable, 64% of teachers, 82% of parents, and 54% of students agreed. 
The majority of students reported that teachers treated them with respect 
(70%), while 76% of teachers reported that most students respected 
teachers. While 67% of students said they enjoy learning at school, 60% 
reported that school is just a boring routine. Most parents (80%) reported 
that their child enjoyed school, and 77% reported that their child was 
eager and enthusiastic about learning at school. 

4. Parent Involvement 

• Teachers reported that they encouraged parents to participate in school 
activities (94%), regularly scheduled conferences (82%), and that parents 
were eager to talk about their children’s academic progress (79%). 

Parents also reported that they were encouraged to participate in school 
activities (84%). Parents felt free to contact teachers (91%), and reported 
that teachers hstened to their concerns (81%). 

5. Grade the School 

• Seventy-nine percent of teachers agreed that their school deserved a grade 
of “A” or “B”. Ninety-five percent of teachers agreed that their school 
deserved a grade of “C” or better. 
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• Sixty-six percent of parents agreed that their school deserved a grade of 
“A” or “B”. Ninety-one percent of parents agreed that their school 
deserved a grade of “C” or better. 

• Fifty-one percent of students agreed that their school deserved a grade of 
“A” or “B”. Eighty-one percent of students agreed that their school 
deserved a grade of “C” or better. 

Open-Ended Results 

Analysis of the open-ended responses was contracted to the University 
of North Carohna at Greensboro both years of the survey. Samples 
large enough to represent the school and selected demographic 
characteristics were selected for transcription and categorization. 

Responses were summarized and provided to each school. Generally, 
the responses to the open-ended questions supported and validated the 
multiple-choice results. Summary reports were sent to each school. 

Use of Survey Results for School Planning 

Each year schools received their multiple-choice survey item data in both 
tabular and graphic form, which schools teams have used extensively in 
developing their School Improvement Plans, required as part of the State 
accountability program. Principals met several times each year with the 
Director of Secondary Achievement, then in turn met with their school teams, 
in order to use survey results for planning. 

Phase 2 of the Evaluation: A Site-Based, Formative Evaluation Cycle 
Now that there are two years of basehne survey data, plans are to administer 
the survey once every three years. Planning is underway to use the 
intervening years to work on improvement in the psychometric qualities of 
the survey instruments, and toward the use of formative, qualitative 
evaluation techniques, as described in the timeline, above. 

Formative, School-Based Evaluation Planning 

Schools were notified in February, 1998 that the following systemwide issues 
had emerged from the surveys: 

• differentiated instruction 

• advisor/advisee programs 

• block/flexible programs 

• team organization and interdisciphnary teaming 

• collaboration between core and encore (elective) teachers 

• intramural programs 

• school safety 

• disciphne 
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• parent involvement. 

School were asked to prioritize these issues in terms of concern from their 
school staffs perspective. With monitoring and oversight provided by 
Guilford County Schools Assessment and Evaluation, teams from the 
University of North Carolina at Greensboro will work with each school on 
designing and carrying out the formative evaluation plan in 1997-98 and 
1998-99. 

Improvement the Psychometric Characteristics of the Survey 
Since the Middle School Survey will not be administered again until 1999-00, 
the intervening time can be spent on improving the survey. To this end, the 
Office of Assessment and Evaluation will conduct item analysis and factor 
analysis in order to pursue the possibihty of reducing the number of items 
and/or eliminating items. Focus groups comprised of representatives from 
the student, teacher and parent groups will work with staff to improve the 
wording and clarity of the items. 

Discussion 

The importance of this evaluation of the Middle School Program has become 
increasingly clear. First, a clear message regarding the importance of the 
Middle School Program was sent to the schools by the Board of Education by 
their mandate that the program be evaluated. The nature of the survey was 
such that both the implementation and the outcomes of the Middle School 
Program were monitored. 

Second, anecdotal reports by numerous principals suggest that both the 
information provided by the survey results, and the planning meetings with 
Central Office staff to discuss the meaning and use of the results, were 
extremely helpful. All of our schools have by state law submitted formal 
School Improvement Plans as one part of the accountability program for the 
last several years. Middle school principals have consistently indicated that 
the planning process involving the Middle School Survey helped a great deal 
in formulating the School Improvement Plans. 

The formative evaluation process just beginning has been positively viewed 
by principals as a way to engage the school leadership team and others in 
continuous improvement. In the midst of a very high stakes accountabihty 
program and the stresses inherent to that process, site-based involvement in 
the planning and evaluation process is both positive and proactive. 

Reference 

George, P. & Alexander, W. (1993). The Exemplary Middle School. (2"^ 
Edition). New York: Holt, Rinehart. 
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Appendix 1 
Appendix 2 



Appendices 

Sample Reports from 1996-97 
Sample Reports from 1997-98 



Middle School TEACHER Survey (Spring, 1997) 
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ALLEN JAY Middle School STUDENT Survey 
(Spring, 1996) 
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